Entropy production in full phase space for continuous stochastic dynamics 
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The total entropy production and its three constituent components are described both as fluctu- 
ating trajectory-dependent quantities and as averaged contributions in the context of the continuous 
Markovian dynamics, described by stochastic differential equations with multiplicative noise, of sys- 
tems with both odd and even coordinates with respect to time reversal, such as dynamics in full 
phase space. Two of these constituent quantities obey integral fluctuation theorems and are thus 
rigorously positive in the mean by Jensen's inequality. The third, however, is not and furthermore 
cannot be uniquely associated with irreversibility arising from relaxation, nor with the breakage 
of detailed balance brought about by non-equilibrium constraints. The properties of the various 
contributions to total entropy production are explored through the consideration of two examples: 
steady state heat conduction due to a temperature gradient, and transitions between stationary 
states of drift-diffusion on a ring, both in the context of the full phase space dynamics of a single 
Brownian particle. 

PACS numbers: 05.70.Ln,05.40.-a 



I. INTRODUCTION 



The concept of entropy was introduced over 150 years 
ago to provide a measure of the evident irreversibility of 
macroscopic thermodynamic phenomena. The conflict 
between its monotonic increase and the underlying time 
reversal symmetry of the microscopic dynamics, first 
pointed out by Loschmidt, is but one of its apparent 
mysteries. Nevertheless, in recent years significant 
insights into the nature of irreversibility and entropy 
production have emerged, partly due to the need for 
a framework to interpret thermodynamic processes on 
the nanoscalc. These developments had their beginnings 
in the dissipation function and Fluctuation Theorem 
in deterministic thermostatted systems considered by 
Evans et al [i|-|j|: ^-nd have continued with similar 
concepts within the realms of chaos theory Q and of 
stochastic dynamical modelling [6|, lZ[. Some powerful 
results such as the Crooks and Jarzynski relations 
stand out [sHiOj along with a unifying framework 
for 



overdamped Langcvin dynamics |ll| based on a 
stochastic description of the first law of thermodynamics 
commonly referred to as stochastic energetics [l2| . In 
short, entropy production is a measure of the relative 
fikelihoods of forward and reversed behaviour within the 
context of a model dynamical framework that includes 
specific dissipative terms. There are certain differences 
in viewpoint, but the central insight is that a mechanical 
(or maybe dynamical) quantity can be defined that 
matches the behaviour of thermodynamic entropy, in 
particular that on average, it increases with time. Its 
fluctuating nature provides additional insight into the 
behaviour of small systems. 

More recently, it was proposed that the entropy 
production associated with non-equilibrium states of 
small systems, arising from an underlying stochastic 



model of the dynamics, could be divided into two com- 
ponents, one related to relaxation (sometimes restricted 
to transitions between stationary states), and the other 
to any fundamental constraint that maintains the system 
away from an equilibrium |13l42l| . The two components, 
termed adiabatic and non-adiabatic production rates 
respectively, were mapped onto earlier concepts known 
as excess and house-keeping heat transfers [13, 1221. 



In a recent development [23[, however, it was shown 



that a third component of entropy production could be 
conceived, arising from the non-equilibrium constraint, 
but associated with relaxation towards the stationary 
state. It only arises when odd dynamical variables play 
a role in the dynamics, and even then only in specific 
cases. Only two of the three components of entropy 
production satisfy an integral fluctuation theorem (IFT), 
making them rigorously positive in the mean, properties 
shared by the sum of all three; the third, however, does 
not satisfy an IFT, and in the mean can take either sign. 

In this paper, we develop these ideas further, within 
a framework of full phase space continuous dynamics 
modelled by stochastic differential equations, with the 
aim of pinning down the specific form of the three 
contributions, both in the mean and in fluctuations 
about the mean and go on to make use of the formalism 
in some instructive example systems. 



II. THREE CONTRIBUTIONS TO ENTROPY 
PRODUCTION 

We begin by considering the dynamics of a general 
set of variables x = {xi, X2, ■ ■ ■ Xn) that may be odd or 
even under time reversal by considering the operation 
ex = (eia::i,e2a;2, . . .e„a;„) where ei — ±1 for even and 
odd variables Xi respectively. Specifically, we consider 



continuous Markovian dynamics described by a system 
of arbitrary uncorrelated Ito stochastic differential equa- 
tions (SDEs) such that the evolution of the coordinates 
X are given as 



dxi = Ai{x, t)dt + Bi{x, t)dWi. 



(1) 



where dWi denotes the Wiener process. Since we allow 
Xi to be either odd or even under time reversal we can 
divide the deterministic dynamics into reversible and ir- 
reversible components such that 

dx, = ^-""(a;, t)dt + ^-{x, t)dt + B,{x, t)dW., (2) 
by defining 



J^l{x,t) = - {A,{x,t) + e,A,{ex)) = e^^^{ex,t) (3) 



Ar(a;,i) = - {A,{x,t) - e,A{ex)) ^ -e,Af''{ex,t). 

(4) 
We briefly note that we intend our notation, A{ex, t), to 
imply a time reversal of all parameters that constitute 
A whether they be dynamical variables included in x or 
not. For example, a term proportional to a magnetic 
field appearing in Ax, where x is an even spatial coordi- 
nate, would form part of ^™^ since magnetic fields are 
odd with respect to time reversal in contrast with, for 
example, a force, F, which would appear in A" since 
force is even with respect to time reversal. 

We have specified for simplicity that all our SDEs 
arc driven by uncorrelated noise such that we have no 
cross derivatives in the corresponding Fokker-Planck 
equation. As such we may then represent the noise 
strengths as diffusion coefficients 



^B,{x,tf ^ D,{x,t) 



(5) 



that appear in a Fokker-Planck equation describing the 
joint probability density function of the coordinates 

dp{x, t) ^ 
dt 

-J^TT iMx,t)p{x,t))+J2jr^ {D,{x,t)p{x,t)) . 



dxi 



dxf 



(6) 



For simplicity in the later development we assume that 
the diffusion coefficient is symmetric with respect to 
time reversal such that D{ex) = D{x), which puts 
no restriction on the dependence on even coordinates, 
but requires that D(x) is an even function of any odd 
coordinates. 

It is helpful to express the Fokker-Planck equation 



as a continuity equation in terms of the probability 
density current J{x, t) 



dpjx, t) 
dt 



= -V ■ J{x,t) 

= -V.(j-(a;,i) + J-^(a;,t)) (7) 



which we separate into irreversible and reversible compo- 
nents. These take vector form J = (Ji, J2, ■ ■ ■ Jn) as do 
the drift and diffusion coefficients A = {Ai,A2,...An) 
and D = (Di, D2, ■ ■ ■ Dn) such that 

J"' (a;, t) = A''{x, t)p{x, t)-V ■ {D{x, t)p{x, t)) 
.r''^x,t)^A'''''{x,t)p{x,t). (8) 

Having set out the dynamics we shall be using, we now 
consider the general procedure for producing quantities 
which obey IFTs. Given an interval of duration r, such a 
quantity consists of a difference between the logarithmic 
probability density of a given trajectory under what we 
shall term the forward dynamics, time dependence of the 
dynamics (equivalent here to an external protocol) and 
initial distribution of starting configurations, and that of 
another appropriately chosen trajectory under suitable 
dynamics, protocol and initial distribution. We write 
p[x] as the probability density of the forward trajectory 
or path, X = x{t) for Q <t <t, with a probability density 
function of starting configurations, p(a;(0), 0), which acts 
as an initial condition for the Fokker-Planck equation 
introduced earlier. A quantity that obeys an IFT is then 
of the form 



A[x]^\n[p[x]/p*[x*]] 



(9) 



where p* [x ] is the probability density of a path, x* , un- 
der chosen dynamics (with specified nature and time de- 
pendence) and initial condition. Demonstrating the ad- 
herence of such a quantity to an IFT with respect to 
the forward dynamics and time dependence is straight- 
forward by the reasoning 

(expM[i]])^/rf5p[5]expM[5]] 
,p*[x\ 



dxp[^- 



p[x\ 



I dx* p*[x*] ~ 1. 



(10) 



Such a result requires a Jacobian of unity for the path 
transformation S — >■ 5* in order for the path integrals 
to be equivalent (a result assured for any involutive 
transformation) and the requirement p*[5*] = for 
all p[5] — ensuring that for normalised p and p* all 
possible paths under the dynamics that produce p* are 
contained within the final integral. This may be seen 
as a more general version of the ergodic consistency 
requirement [2J] since it is a condition required for 
the appropriate inclusion of paths in the final integral 
of Eq. pI7|) . but we do not demand that p and p* be 



non-zero for all paths. Further, any quantity A[x] based 
on a transformation with a Jacobian of unity takes the 
same form irrespective of whether probabilities or prob- 
ability densities are used in the construction owing to 
the equivalence of measure. Such a quantity is therefore 
constructed unambiguously and is the direct analogue of 
the same quantity defined in discrete space [1^, [23 . The 
implication of the positivity in the mean, (A[5]) > 0, 
of the quantity A[x] is assured by Jensen's inequality. 
We point out that the path, x* , and the dynamics must 
be carefully chosen in order to satisfy the requirements 
and to produce a physically meaningful quantity. This 
is particularly relevant in the presence of both odd 
and even variables as for many physical systems the 
common choice of reverse path x* (t) = x{t — t) cannot 
be generated under the forward dynamics, and if used 
to define a quantity of the form in Eq. ([9|) would render 
the final integral in Eq. pH)) equal to zero. However, the 
choice x*{t) — ex{T~t) typically can be generated and 
so leads to an IFT. However, this choice of the second 
path, x*{t) = ex{T~t), is appropriate not just because 
of the guarantee of an IFT, but because it means the 
constructed quantity A[x] serves as a measure of the 
irreversibility of the process and thus characterises the 
total entropy production. 

By following the above rules, and making the definitions 
x^{t) = ex{T — t), a^(i) = x{T — t) and x'^{t) = ex{t), we 
may construct dimensionless entropy changes, which are 
thermodynamically meaningful when multiplied by ks, 
of the form in Eq. (jH]): 



A5tot = lnp[5]-lnp«[ST] 



Rr;sti 



In 



p{x{t),t) 



In 



p[x{t)\x{0)] 
p^[ex{0)\ex{T)] 



(11) 



AS*! 



= ln 



lnp[x] 
p{x{0),0) 



Inp-'^'^i^] 



In 



p[x{t)\x{ 



p{x{t),t) p'''^''^[x{0)\x{t)] 



(12) 



AS2 = l-ap[x]-\np'"^[x'] 



In 



In 



p[x{t)\x{0)] 



p{x{0),0) 

p{x{0),0) ' ^" p^'^[ex{T)\ex{0)] 



(13) 



such that the total path probability densities are divided 
into initial probability density distributions and condi- 
tional probability densities. In the above the label R des- 
ignates a reversed protocol, equivalent here to reversed 
time dependence in the dynamics, and 'ad' designates 
that the dynamics are so-called adjoint with respect to 
the forward dynamics, defined as the dynamics which 
reach the same stationary state, but with the opposite 
stationary current [151, [l^ [2^ . All are expected to obey 
IFTs by the nature of their form. Then, by the construc- 



tion AS'tot = AS*! + AS2 + AS3, we define 

A53 == lnp'*'^[#] + lnp^'^'^[x^] - lnp[x] - lnp^[x^] 



In 



p^^^"[a;(0)|a;(T)K^[£a;(r)|ea;(0)] 
p[x{t) I x{0)]p^ [£a;(0) |£a;(r)] 



(14) 



which cannot be expressed in the form of Eq. © and 
so does not obey an IFT. By following the formalism of 
Seifert [lll,[ll we identify 



ASt, 



In 



p{x{0),0) 
p{x{t),t) 



t=o 



AQ 



kBT,nAx{t),t) 



= AS*. 



sys 



AS", 



incd : 



(15) 



where AS'sys is known as the change in dimensionless 
system entropy and AS'mcd is a generalisation of the di- 
mensionless entropy production in the environment, or 
medium, for an environmental temperature, Tcnv, which 
we allow to be phase space or time dependent (a prop- 
erty we employ in example I). The integral should be 
interpreted as a total medium entropy change, consid- 
ered as the sum of all the incremental heat transfers 
to separate fixed temperature heat baths to which the 
particle is exposed over the course of its trajectory, di- 
vided by the appropriate temperature. Of course, this 
reproduces the usual AS'mcd = AQ/ksTcnv of stochas- 
tic thermodynamics |ll| when the temperature is con- 
stant. Furthermore, to make connection with a key con- 
cept in non-equilibrium thermodynamics, we may divide 
the heat transfer to the environment (considering one 
at a constant temperature for clarity without loss of 
generality), into the so-called excess and house-keeping 
heats according to the formalism of Oono and Paniconi 
[23 | such that for a given environmental temperature 
AQ = AQcx + AQhk- To align our quantities with such 
a formalism we associate ASi with the excess heat 

AQex - (ASi-AS',y,)fcsTe„v, (16) 

AS'2 with a so called 'generalised house-keeping heat' [23| 

AOhk,G = ASsfcsTenv (17) 

and AS3 with the 'transient house-keeping heat'. 



AQhk,T == ASsksTc, 



(18) 



named to reflect its mean behaviour, such that AQhk = 
AQhk.G + A(5hk,T- Like previous formalisms [1^ [21| 
where the entropy production was divided into two con- 
tributions associated with relaxation, and an absence of 
detailed balance, respectively, both in the mean and in 
detail |19l - [2l| . we have a contribution ASi which is non- 
zero only in the presence of relaxation and A 5*2 which is 
non-zero only in the absence of detailed balance both in 
the mean and in detail. However, we also have a quantity 
AS3 which is non-zero in detail only in the absence of de- 
tailed balance, but only contributes in the mean during 
the course of relaxation. Such a formalism asserts that 



the two origins of entropy production may often be more 
closely related, with such a circumstance arising under 
the inclusion of odd variables and when the stationary 
distribution is asymmetric in any of those odd variables. 
The aim of this paper is to derive the equations of mo- 
tion for each of these quantities for continuous stochastic 
systems and to illustrate their behaviour through some 
simple examples. 



III. REPRESENTING ENTROPY 

PRODUCTION FOR CONTINUOUS 

BEHAVIOUR 

Since we are describing the dynamics using SDEs it is 
sensible to seek a description of a small increment in each 
entropy production given an increment in the underlying 
variables a/ — a; = x{t + dt) — x{t) in a time dt so that we 
identify from Eqs. ([TT|) . ([T^ . (fH)) and ([Ti l) 



In 



dAS'tot = -(i(lnp) 
dAS"! = -d(lnp)+ln 
dA^a = In 



p{x' ,t + dt\x,t) 

p{ex,t + dt\£x',t) 

p{x! ,t + dt\x,t) 

p'''^{x,t + dt\x/,t) 

p{x',t + dt\x,t) 



p^'^{£Xf,t + dt\£X,t) 



dASa = In 



p'^'^ix^t + dt\x! ,t)p'"^{£xf ,t + dt\£x,t) 
p{x', t + dt\x, t)p{ex, t + dt\£xf, t) 



(19) 
(20) 
(21) 
(22) 



thereby establishing the SDEs that describe en- 
tropy production and noting the abbreviation 
d{\n{p)) = \n{p{x{t + dt),t + dt)/p{x{t),t)). 



To proceed we require a representation of the path 
probabilities in these expressions valid over the small 
time interval dt. This may be achieved by considered the 
short time Green's function or 'short time propagator' 
[2a | which is given generally as the conditional probabil- 
ity of a displacement dx ~ x' — x in & time dt subject to 
a delta function initial condition and is of the form 



p{x!,t+dt\x,t) = Yl 



1 



ATrD,{r,t)dt 



xexp 



{dxi-A,{r,t)dt+2a{dD,{r,t)/dri)dtY 



—adt 



AD,{r)dt 
dA,{r,t) , ^^d'D,{r,t) 



dn 



a^dt- 



drf 



(23) 



where dxi = x'^ — Xi and where a is a free parameter 
ranging from to 1 which defines the evaluation point 
of certain terms in the propagator r= ax! + (l — a)x and 
r, = axi' + {1 — a)xi, and which reflects the ambiguity 
of a discretised interpretation of continuous stochastic 
behaviour. We note, however that as di ^- all forms 
for the propagator arc correct: they are all accurate to 
first order in dt and result in the same Fokker-Planck 
equation. 

We wish to construct the increment in entropy 
production in the medium by a consideration of 



dAS", 



nicd 



In 



p{a^,t+dt\x,t) 
p{ex,t + dt\ex',t) 



(24) 



by employing the appropriate reverse short time propa- 
gator 




47rA(er',i)di 



exp 



{~e,dx, - A{£r',t)dt + 2h{dD,{erJ ,t)ld{e,r',))dtf _ ^^^ dA,{erJ ,t) ^ ^^^^d^D^ier' ,t) 



AD,{£r')dt 



^{e^r[) 



dieyd 



(25) 



where & is a corresponding free parameter ranging from to 1 such that r' = 6a;+(l — &)a/ and r[ = bxi + {l — b)x'i. 
Using Eqs. ^ and (HI along with the assumption Di{£x) — Di{x) such that any diffusion constants for odd variables 
are symmetric, we may write 



p{£x, t + dt\£x', ^) = TT 



■ exp 



{~e,dx, - £, (-Ar(r',0 + Af{r',t)) dt + 2h{dD,{rJ ,t)/d{e,r[))dty 



4TTDi{r',t)dt 

'de,A'^{r^,t) de,Al'^^r^,t) 



-bdt 



d{eir[) 



d{£ir'i) 



4:Di{r')dt 



.2,u d'D.{r',t) 

r\2 



¥dt- 



d{e..r[) 



(26) 



which is the same as 



p{ex, t + dt\sx' , i) = TT 



'dAf{T^,t) 



-dx, - (-^"-(r', t) + Af(r', t)) dt + 2h{dD,{r',t)/dr[)dty 



-bdt 



dr 



dr'- 



ADi{r')dt 



,^^d^D,{r^,t) 



Vdt 



dr': 



(27) 



The mathematical details necessary for the development 
of Eq. (HH), which due to their somewhat cumbersome 
nature we leave to appendix \^ reveal that for multi- 
plicative noise one obtains a result which is dependent 
on the choice a and b. The resolution of this apparent 
arbitrariness is not related to the nature of the underly- 
ing SDEs, but rather on consistently using the equivalent 
evaluation point for forward and time reversed paths on 
an infinitesimal scale. The normal rules of calculus would 
dictate no dependence, but different rules apply to SDEs 
and stochastic calculus. Such a consideration reveals the 
correct choice in Eq. (j27p to be 6 = 1 — a with a remain- 
ing as a free parameter. This yields the unambiguous Ito 
SDE for the medium entropy change 



sr-^n^), Ar{x)j^nx) ^^ 

dAS^.., = ^ -^^dx. - -^^^-—dt 



^4^dt-^^^dt 

oxi axi 

Di{x) dxi 

{A'^^x)^A"{x))dD,{x) ^^ 
Di{x) dxi 

d'^DAx) , 1 /dDAx)\\ 

-dt + --^^ ^ dt 28 

D^{x) \ dx^ J ^ ' 



dxf 



where for brevity we use notation f{x) = f{x,t). 
To clarify, in such an approach choices may include 
Stratonovich (a = b = 1/2) evaluation for both prop- 
agators in Eq. (|24^. a choice which is implicitly used 
by many authors |15l . |25| within integrated Onsager- 
Machlup approaches, but does not preclude others in 
the construction of SDEs such as, for example, an 
Ito prescription (a ~ 0) in the forward propagator 
and a Hanggi-Klimontovich (6 = 1) in the backwards 
propagator. We point out that all evaluation points 
lead to the correct path probability when supplemented 
with the correct multiplication scheme, but that if one 
has multiplicative noise, the correct representation of 
the entropy production requires the more exact relation 
between the evaluation points. 

Proceeding, we may now construct an SDE for the 
total entropy production by first considering an in- 
crement in the system entropy which under Ito rules 



dASsys = -d{lnp{x)) 
1 dp{x) 



p{x) dt 



dt- 



-y 



dp{x) 
p{x) ^—' dxi 



dxi 



E 



p{x) \ dxf p{x) \ dxi 



dt. 



(29) 



which together with Eq. ([^5)) , and after insertion of the 
Fokker-Planck equation, leads to 

,.c, _Y- 1 Mx), , 1 d{A{x)pix)) 



p{x) dx^ 



p{x) dxi 



-dt 



1 ( d\D.,{x)p{x)) f.dM^ 



p{x) 



dxf 



Di{x) ( dp{x) 



p{x) \ dx. 



dxf 



dt 



Af (a;)^^ _ A^<^^x)Af{x) ^^ 



Di{x) 



Ddx) 



^dt-^^:^dt 

oxi oxi 

1 dD,{x) 



-dxi 



Di{x) dx 
{A'''^{x)-A"{x))dD,{x) 



D,{x) 



dxi 



dt 



"'^'(^^^t+^r^^M) dt. 



dxf 



Di(x) 



dx,. 



(30) 



If Stratonovich rules, for example, are preferred we can 
write (by definition of the Stratonovich integral, indi- 
cated by the o notation) 

^AQ V^ 1 ^^(^) w M 1 d{Aix)pix)) ^^ 
dAAtot = > r^^^ ° dxi + ^-^ dt 



p{x) dx^ 
1 ( d\D,{x)p{x)) 



p{x) 



dxf 



p{x) dx. 



dt 



^^x) Ar{x)j^ax) ^^ 
o dxi „ , , dt 



D,{x) 



A (a;) 



n,^.,^(SM^it 



dxi V Di{x) 



1 dD,{x) 
D,{x) dxi 



> dxj 



(31) 



This is a very general and robust definition of the 
entropy production for continuous stochastic behaviour 
and can be thought of as a generahsation of the pio- 
neering approach in [ll| wherein the equation of motion 
for entropy essentially describes dAS'tot for a specific 
system with additive noise, even variables {ex ~ x) and 
implicitly using Stratonovich rules. 

We point out that such a construction allows us to 
consider purely deterministic coordinates {Di{x) = 0) as 
would apply, for example, to the case of spatial coordi- 
nates within a full phase space Langevin description. In 
such coordinates Di{x) is assumed constant and taken 
to zero. The remaining terms then clearly diverge unless 
we demand A"{x) = since in these instances, for the 
reverse path to be a solution to the forward dynamics 
the motion must be purely reversible. This condition 
simply amounts to the requirement that the reverse 
path exists. There is, however, a contribution to the 
medium entropy production, due to the dynamics of 
these coordinates, technically since path probability 
densities, not probabilities, are being considered in the 
formulation. The contribution to the medium entropy 
production due to the deterministic behaviour of these 
coordinates is 



AS, 



mcd.dct 



dxi 



dt. 



(32) 



a result that provides an insight into the similarities and 
differences between stochastic and deterministic mea- 
sures of irreversibility: it is demonstrably equal to the 
phase space contraction found in non-linear dynamical 
systems, which is associated with the heat transfer to the 
environment brought about by thcrmostatting terms in 
such approaches. This leads to a quantity that is positive 
in the mean for deterministic systems: the dissipation 
function [2|. We point out, however that total entropy 
production, as defined here for stochastic systems, is zero 
for deterministic dynamics. This is because the change 
in the system entropy would be equal and opposite 
to the change in medium entropy, technically since it 
involves probability densities at the start and end of 
the process. In contrast the dissipation function can 
provide a measure of irreversibility because it involves 
a comparison of trajectories originating from the same 
starting distribution. This contrast is to be expected as 
the total entropy production, as defined for the systems 
we consider, arises from explicit irreversibility in the 
dynamics, which deterministic, reversible equations do 
not provide. 

Frequently the average entropy production rate is 
argued to be proportional to the mean probability 
flux squared, as derived, for example, by taking the 
time derivative of the Gibbs entropy of a system, and 
identifying an evidently positive contribution as the 
total entropy production rate and the remainder as 
the (negative of) the medium entropy production rate 



[2ll |27| . We prefer however, to derive the average 
contributions directly from the SDEs so that we can 
avoid arbitrarily identifying a positive contribution 
with a quantity expected to obey an IFT: strictly there 
is no guarantee such a division is unique, as another 
description shows [2^. To do so is straightforward and 
requires us to find the average increment in A 6*101 by 
means of the integral 

(dAS'tot) ^ dx dx' p{x, t)p{x', t + dt\x, t)dAStot- 

(33) 
The benefit of such a formulation is that we may charac- 
terise dAS'tot using an Ito SDE based on the underlying 
relations dxi = Aidt + BidWi and then use the martin- 
gale property of the Ito stochastic integral {BidWi) = 
since Bi is non-anticipating, such that we can simplify 
the integral in Eq. (j33|) by writing 



(34) 



(dAS-tot) = / dxp{x){dAStot\x) 



and evaluating the conditional average (dAS'totla^) by 
replacing all occurrences of dxi with {Af + Af^^)dt in 
dAS'tot- We thus get 



(dA5tot) = J2 



dx^^m^+2pix)'^^^ 



D,{x) 



dxi 



^ A"{x)da,{x) .d^D,{x) 



+ 



Di{x) dxi 
p{x) (dD,{x) 



Di{x) \ dx 



dxi 
d^D,{x)pix) 
dxf 



-D,{x) 



d'^p{x) Di{x) ( dp{x) 



dx'j 



p{x) 



dxi 



dt. (35) 



By applying the product rule, integrating by parts and 
assuming p{x) and dp{x)/dxi cither vanish or cancel at 
the boundaries, we may simplify to find the total entropy 
production rate 

d{AStot) 



dt 



Y^jdx 



,dDdx) 



p{x)jVf{x)-D,{x)^-pix)^ 



p{x)Di{x) 



or more concisely 

d(A^tot) 



dt 



Y^jdx 



p{x)D,{x)' 



(36) 



(37) 



providing an expression for the mean instantaneous en- 
tropy production rate which is rigorously positive, as it 
must be because of the adherence of A5'tot to an IFT, 
and is dependent on the irreversible fiux. 



IV. EXPRESSIONS FOR ASi, AS2 AND AS3 



In order to consider a division of the entropy produc- 
tion into the thermodynamically meaningful quantities 
outhned above we are required to construct path proba- 
bihties using the so-called adjoint dynamics. These dy- 
namics may not be physically realisable: for example 
they may require negative positional steps to result from 
positive velocities (as indicated by the paths a^{t) and 
x^{t)), but this is of no concern since they are only intro- 
duced for the mathematical construction of the entropy 
contributions. We consider an arbitrary stationary distri- 
bution of a given system which may be written in terms 
of a non-equilibrium potential, (f>{x), such that 



/'(a;)=exph0(a;)] 



(38) 



and assert that the adjoint dynamics are those that result 
in the same stationary distribution, but have an opposite 
flux. As such we require 



dt 



-V -r^x) = V- J"*'''^(a;) =0 



(39) 



with 



.r 



st.ad 



(x) = -rHx). 



(40) 



In order to characterise the adjoint dynamics we con- 
struct the adjoint flux according to 



= A^'*(a;)e-"^(^) - 
\{x) 



d 

dxi 
d 

dxt 
dD,{x) 



'D,{x)f'{x)) 



Dt{x)e 



dxi 
dD,{x) 



dxi 



D,{x) 
-D,{x) 



d(j){x) 
dxi 

d(j){x) 
dxi 



-4,(x) 



-0(x) 



Consequently we have the requirement 



^''^(a;) ^-A{x) + 2 



dD,{x) 



-2D,ix) 



d<j)[x) 



dxi dxi 

Let us now consider the quantity 

p{oc!,t + dt\x,t) 



dAS'rx = In 



p'''^{x,t + dt\x',t)' 



(41) 



(42) 



(43) 



where AS'ox = AQcx/fcsTcnv, which we have previously 
asserted constitutes part of the incremental contribution 
to the quantity AS*! based on relations in Eqs. ([T^ and 
its short time representation. We evaluate Eq. (P^l) . tak- 
ing the transition probability density in the numerator 
from Eq. ([^5]) and, for convenience, choosing a = 1/2. 
We can represent the transition probability density ap- 
pearing in the denominator through a similar construc- 
tion, but using a substitution for the adjoint drift term 
from Eq. (|42|). together with the complementary evalua- 
tion point choice 6 = 1 — a = 1/2 such that 



p'"^{x,t + dt\x',t) = Y[ 



1 



ATTDi{r)dt 



exp 



i-dx, + {A,{r) - 2{dD,{r)/dr,) + 2D,{r){dcj){r)/dn))dt + {dDi{r)/dr,)dtf 



AD.,{r)dt 



tI-(-''^)-^--<'>1^)-I^ 



(44) 



I 

Since we have in both cases chosen evaluation at a = the ratio of these two propagators we find 
b = 1/2 we note that multiplication follows Stratonovich 
rules so that we have f{r)dxi — f{x) o dxi. Considering 



dASc: 



In 



p{x! ,t + dt\x,t) 
p'^<^{x,t + dt\x!,t) 



Y^D,{x) 

i 

d(j){x) 
dxi 



f dcp{x) 
\ dxi 



dt + Ai{x) 



d(j){x) 
dxi 



dt 



o dxi 



dAAx) , d^D,(x) , 



dxi 



dxf 



D.ix)'^mdt^2'-mmdt. 



dxj 



dxi dxi 



(45) 



However, we also have the condition 

_d_ 
dx 



Et)^-!^ 



A.(.)-^^ + A:(a.).^'^("^ 



dxi 



dxi 



-M.f-^-D.i.)('^^^'^y ''^^^^^ 



d^D^jx) 
dx'j 



dxi 



+ D,{x) 



dx. 



dxi 



9^^ 9A(^a0(^\ _^(,) 



dx. 



dxj dxi 



(46) 



and so by insertion we arrive at 



ln4^:ii±^5A^ = ^-M^odx, (47) 



p'''^{x,t + dt\x!,t) 



dxi 



which justifies the usual characterisation of the adjoint 
dynamics [1^, [I^, [SJ] for use in continuous dynamics 
when written 



p{x! ,t + dt\x,t) p"*(a/) 



p''<^{x,t + dt\x!,t) p^^{x) 



(48) 



through consideration of Eq. ([55]) and the Stratonovich 
rules which mimic normal calculus. 

We construct an increment in A^i, using the above 
result with the inclusion of a change in system entropy 
such that 

dASi = dASsys + d (AQox/fcsT'cnv) 
= -d(lnp) - 2^ — o dXi 



dxi 



1 dp(x) 



dt-E 



1 dp{x) 



dxi 



1 d^p{x) 1 f dp{x)^ 



p{x) dt ^—' p{x) dx 

\pix) dxi {p{^)Y V dxi 

d(t>(x) , ^ , ,d'^(t>{x) , 
-^7^^dx^ - D,{x)—p:^dt. 
dxi dxj 



dt 



(49) 



Applying the same averaging procedure used to calculate 
{d AS tot) wc find 



{dASi) = y I dxp{x)^^p^dt 
^—^ J dxi 



dA,{x)^^ d^D,{x)p{x)) 



D^{x) f dpjx) 
p{x) \ dxi 
d(j){x) 



dxj 
d'^p{x) 



dt 



dt — DAx) — ^ „ dt 
dxf 

d'^(t>{x) 



p{x)A{x) ;;' ' dt-p{x)D{x) ^\ dt. (50) 



i9x. 



dxj 



However, using Eq. (|46)) we may represent this as 

(dASi) = 



^'^") ^^^(")^^t+M.)A(a.)('^Vdt 



p{x) \ dxi 



dD,{x) dcj){x) 



dxj 



d^(t){x) 



.d^Ddx^ dHiMx)p{x)i , ^ ..-. ,, 

-Pi"") ^^2 dt —^ dt - D,{x)^-^dt. 



dxj 



dxj 



, d^p{x) 
dxj 



(51) 



By further integration by parts, dropping boundary 
terms and rearranging, this becomes 



{dASi)^Yl f dx — 



D,_{xi r dpjx) 

(x) \ dxf 



dt 



+ p[x)D,{x)i'4$\ dt + 2Ddx)'4^'4^dt 



V dXi 



dxi dxi 



(52) 



which can be written 

d{ASi) _ d{ASsys + (AQex/fcsTenv)) 



dt 






dt 
p{x) fj,{x) Jf{x) 



(x) \ p{x) p^^{x) 



st.ir / 



pjx) Ji^jx) jr^jx) 

Di{x) \ p{x) p^*(a;) 



(53) 



assuring the positivity of such a contribution. Since it 
can be written in terms of the total current Ji in this 
way, it maps precisely onto the non-adiabatic entropy 
production appearing in |2l| and thus can be expressed 
as 



diASj^^_r^j_p^^^pix) 



dt 



dt p^^(x) 



(54) 



as highlighted by the authors of [2l[. We however 
emphasise that Eq. (|54l) is to be considered alongside 
the accompanying SDE in Eq. (|49p . from which it has 
been derived directly, rather than by a division of an 
observed positive contribution to the mean rate of 
change of Gibbs entropy into presumed unique transient 
and stationary terms. 

Wc may now by similar means consider an incre- 
ment in A52 as follows: 



dAS2 = In ■ 



p{xf,t+dt\x,t) 
p^'^{ex!,t+dt\ex,t)' 



(55) 



In this case the construction of the denominator follows 
slightly different rules since, unlike A^tot and A^i, the 



alternative path, x , is based on a time reversal of the 
coordinates, but otherwise follows the sequence of the 
forward path. As such b behaves in the same man- 
ner as a rendering r'^ ~ hx\ + (1 — 6)a;i, f{r^)dxi = 
f{x)dxi + 2bD{x)d f {x) / dxidt, as detailed in appendix 
Rl In this case the appropriate choice for the equivalence 
of evaluation points r' = r is a = 6. For continuity, we 

I 



may once again choose a = h = 1/2 with Stratonovich 
multiplication rules: we represent the transition proba- 
bility appearing in the numerator through Eq. (P^ . and 
the denominator by a similar means using the drift term 
given in Eq. (|42|) and the path choice x^{t) = exit), such 
that 



„ad 



{eai ,t + dt\ex,t) = J| ^ 



X exp 



A-KDi{er)dt 
{e,dx, + {A,{er) - 2{dD,{er)/d{e,n)) + 2D,{er){dcf,{er)/d{e,r,)))dt + {dD,{er)/d{e,n))dtf 



A.ier) - 2 ^ /^ / + 2D,{£r) 



2 de. 



AD,{er)dt 
d(j){er)\ dtd^D^{er) 



d{ein) 



d{e,n)) 4 d{e,nf 



(56) 



We can utilise the usual transformation rules and assumptions for ^"', A^^^ and Dt and express d(f>{£r)/d{eiri) — 
eid(j){er)/dri = (fi'^ier) (along with d'^(f){er)/d{eiriY = (j)'-{ex)) such that wc can write the propagator as 



p'"^{ex',t + dt\ex,t) = J| , 



1 



ATTD,{r)dt 



X exp 



{dx, + (Afir) ~ Ar (r) - 2{dD,{r)/dn) + 2e,D,{r)^[{er))dt + {dD,{r)ldn)dty 



AD^{r)dt 



2 On 



dri 



4 9rf 



(57) 



Constructing the ratio in Eq. (f55|) we find 



d^S2 = ^ - 

i 

dJ^(x, 



^i^ATi^^^m^^ 



A (a;) 



D^{X) 



dxi 



dt + D,{x){(l)[{ex)Ydt ~ 2e, 



dD,{x) ^, 



dxi 



4>[{ex)dt 



- sMH^) - Ar{xWA£x)dt + ^±li^^£Mdt 



-f ei(j)'^{ex) o dxi 



1 dP.jx) 
Di{x) dx 



Di{x) dxi 



o dxj 



dxf 



dt~ Di{x)(f>'l{ex)dt 



(58) 



I 

which is the same as 
dA^2 = d {AQi,i,.G/kBT,, 



E 



A^{x)Ar{x)^^ , A^ix) 



A (a;) 
+ ei4>i{ex)dxi - 



-dt ^ ^ , , dxi 

1 dD,{x) 



Di{x) dx 
1 fdD,{x]"-^ 



A (a;) 
dxi 



Di{x) \ dxi 



dt + D.,{x){4>'^{ex)fdt 



2e,<j>'^{£x)^-^^dt + s,iA^ix) - A'rixMiex)dt 



dx 

iAfix)~Al-^x))dD,ix) 



dt. 



Di{x) dxi 

By employing the averaging procedure wc find 

d{AS2) 



(59) 



dt 



Y. h 



P{x) 

A (a;) 



A^ix) 



dD.,{x) 
dxi 



+ £,A(a;)0-(ea;) 



(60) 
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which may be written 

d{AS2) _ d(AQhk,G/fci3Tenv) 



dt 



dt 



E 



dx 



p{x) Jr\ex) 



Di{x) \ p^*-{ex) 



(61) 



Such a form illustrates the positivity requirement of AS'2 
in the mean, resulting from its adherence to an IFT, 
and again Eq. ([6T|) is to be considered alongside the 
complementary SDE in Eq. ([59]). Since it is based on an 
integral over the stationary irreversible flux, d{AS2)/dt 
describes a contribution to entropy production which 
arises from an absence of detailed balance and is non-zero 
both in and out of stationarity. This quantity is to be 
contrasted with the adiabatic entropy production in |2l| 
which we may now consider to be a special case when 
there are only even variables in the dynamics. We point 
out again the importance of the direct derivation of this 
result from the SDE in this formalism, as opposed to a 
division of the irreversible flux into terms with structure 
based solely on p^^{x), which would not have obviously 
led to the above expression. 



We note that the integral in Eq. (|6T|) must reduce 
to the total entropy production, and thus an integral 
over the stationary irreversible flux (i.e. without the e 
factors inside the squared term in the integrand), in the 
stationary state, but there are other circumstances when 
this correspondence applies more generally. A first case 
is when the irreversible stationary flux is proportional 
to the stationary distribution, which would be the case 
for a non-equilibrium constraint that is independent 
of the phase space variables, as illustrated later in 
example II, and a second case is when the total flux in 
each coordinate is everywhere zero {Ji{x) = 0), such as 
for independent variables, Xi, defined on regions with 
natural or reflecting boundaries. 

To complete the description of all three contributions 
to entropy production we now consider an increment in 
A5'3. By using the definition in Eq. p^ 



dASs = In 



p^^(£a/, i + (itiea;, t)p^^(a;, t + (it|a/, t) 
^(a/, t + dt\x^ t)p{£x, t + dt\exf, t) 



(62) 



together with the previously used propagators, and em- 
ploying the stationarity condition evaluated at ex: 

V -r^ex) =0 



dA\''''{ex) 

d{eiXi) 
d^D.jex) 

d{£iXiY 



D,{exM{ex)f 
+ D,{ex)^'l{ex) + 2^^^<j^',{ex) ) e 



wc find 

(iAS'3 = d (AQhk.x/fcsT'cnv) 

= 2, 't'ii^) ° dxi — ei(j)'^{ex) o dxi 

i 

^ Vln ^^P[~'^('^)] exp[-0(£a/)] 
^ exp [-(j)(x:)] exp [-<f){ex)] ' 

which maps onto the same quantity derived from a mas- 
ter equation approach [2j|. We can then construct the 
average contribution by converting to Ito form and per- 
forming the path integral such that 

(dAS'3) = ^ I dxp{x)A,{x){(t)[{x) - e^cj)[{ex))dt 

+ p{x)D,{x){(j)'^{x) - (j)'l{ex))dt (65) 

and proceed to manipulate by integrating by parts, as- 
suming the probability density and current vanish or can- 
cel at boundaries, such that 

(dA53) - Y. f dxp{x)A,{xM{x) - e,c^[{ex))dt 

- J dx — ipix)D,ix)) (0Ka;) - e,^',iex))dt 
= Y^ dx (0^(a;) - e^(j)i{£x)) 

i 

X (a^{x)p{x) - — {p{x)D^{x))] dt 
— 2_] / dx [(j)'i{x) — ei(l)'i{ex)) J^{x)dt 

i 

dx (0(a;) - (j){ex)) -^dt 



dxi 



-Jdx{^ix)^Hex))(^Y.^^) 



dt 



dx (0(a;) - (f){£x)) (V • J{x)) dt. (66) 



By substituting the original Fokkcr-Planck equation we 
may also write this as 



diASs) _ d(AQhk,T/fci3renv) 



dt 



dx 



dt 
dp{x) 
dt 



{c^{x) - cp{ex)) 



= -/d.M^ln.^"(") 



dt p^^{ex) 



(63) 



(67) 



This has a form similar to Eq. ()54|) and is clearly a con- 
tribution to the mean total entropy production rate that 
behaves transiently in a manner similar to AS'i. The 
quantity AS'i appears in the Hatano-Sasa relation which 
describes the entropy production associated with a tran- 
sition between different stationary states. However, in 
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light of Eq. (|67)) wc suggest that ASi, and thus the 
Hatano-Sasa relation and non-adiabatic entropy produc- 
tion, do not represent the entire entropy production as- 
sociated with transitions between stationary states (or 
more generally relaxation) since, in the mean, we can con- 
struct a new quantity which comprises all contributions 
which arc non-zero only during relaxation, by combining 
Eqs. dSH) and ^7^ giving 



d{ASi + AS3) d{AS,y, + (AQex + AQhk.T) /A^sTenv) 



dt 



dt 



dt 
p{x) 



(68) 



This describes a contribution to the mean entropy 
production rate which occurs when the system is out of 
stationarity, but it does not obey an IFT and thus has 
no guarantee of positivity. 

Our central results therefore are expressions for 
three contributions to entropy production for arbitrary 
systems with odd and even dynamical variables evolving 
according to Ito SDKs with multiplicative noise. These 
expressions apply to individual trajectories (Eqs. pip. 
(gHl), dM]) and dM])) and in the mean (Eqs. (|571) . 
((53| . (|6T|) and (|67)) ). Such a demonstration shows 
the additional complexity introduced by the inclusion 
of odd variables if one insists on considering entropy 
production to be due to relaxation or to non-equilibrium 
constraints with particular reference to Eq. (j68|) . One 
may think of (AS*! + AS^) as describing a transient 
contribution to entropy production in the same manner 
as (AS"!), but with the further specification of the nature 
of the coordinates: the entropy production depends 
on whether the variables being described are odd or 
even. The additional complexity of A 6*3 arises because 
Eq. (|68l) can only differ from Eq. (JM]) when the station- 
ary state is out of equilibrium, such that p^*(a;) ^ p^^{ex). 

We of course expect and require that the contribu- 
tions detailed here are related such that 



d(AS'tot) _ d{ASi) d{AS2) diASs) 



dt 



dt 



dt 



dt 



(69) 



yet their forms derived above do not obviously lend them- 
selves to such a demonstration immediately. For com- 
pleteness this is shown in appendix [Bl 



V. EXAMPLE I: STATIONARY HEAT 
TRANSPORT 

We provide as a first example of usage of the above 
formalism a physical situation which necessitates the use 
of odd variables in order to describe entropy production 
adequately: heat transport due to diffusion in one spa- 
tial dimension in the presence of a spatially dependent 
temperature field. Mathematically this system may be 



modelled without odd (velocity) variables by employing 
the overdamped limit and constructing a multiplicative 
SDE and Fokker-Planck equation of the form 




and 

dp{x, t) 
dt 



dx 



d f F{x)p{x,t) 
dx \ my 



dW 



(70) 



+ 



d^ f kBT{x)p{x,t) 
dx"^ \ my 



(71) 
where m is the particle mass, 7 the damping coefficent 
and F{x) the force operating on the particle. We note the 
Ito form of both (for a discussion of the resolution of the 
Ito-Stratonovich dilemma in this case see, for example, 
[29l - l3l| ). This Fokker-Planck equation has a stationary 
distribution 



f\x) 



J\fmy 
ksTix) 



exp 



dx' 



F{x') 



keTix') 



(72) 



where A/" is a normalisation constant. We can quite read- 
ily identify the terms A^^ = F{x)/my, ^™^ = and 
Dx{x) = kBT{x)/my. However, when we come to con- 
struct the dimensionless entropy production in the sta- 
tionary state from Eq. ((3T|) as 



d AS tot 
Dx{x) 



o dx ~ 



F{x) 
kBT{x) 
1 



p^^{x) 
= 



1 dD.,{x) 
Dx{x) dx 

1 dT{x) 
' T{x) dx 
1 dT{x) 



o dx 



1 dp^'^x) 
pP'^ix) dx 



T{x) dx 



f\x) 



kBT{x) 



p^\x) 



o dx 

o dx 
(73) 



we find that there is zero entropy production for all tra- 
jectories. This may be understood either physically by 
recognising that in the overdamped limit one demands 
that the velocity distribution relaxes instantaneously 
thereby preventing any heat transfer due to temperature 
inhomogeneities, or geometrically by recognising the 
impossibility of having stationary flow, and thus entropy 
production, for a system in one dimension with natural 
boundaries. 

To provide a satisfactory representation and to un- 
derstand the entropy production in such a system 
we need to consider the more realistic underdamped 
dynamics in full phase space where we retain both 
position and velocity coordinates, x and v, which are 
even and odd under time reversal, respectively. The 
SDEs and Fokker-Planck equation are now given as 

dx — vdt 



F(x) 
dv = -yvdt H ^—^dt 



m 



2kBT{x)y 



dW 



(74) 
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and 
dp{x, V, t) 



dp{x, v,t) 



dt dx 

d ffFjx) 
dv \\ m 



— ^v 1 p{x, v,t)\ + 



kBT{x)-fd'^p{x,v,t) 



dv^ 



(75) 



Wc may then identify the terms A" = 0, A'^^'^ ~ v, A" = 
--fv, Al''^ = F{x)/m, D^ ^ and Dy = kBT{x)-f/m. 
By Eq. (j3ip the dimensionless entropy production is 



dA^tot 
= —d{lnp{x,v,t)) — 

= —d{\iip{x,v,t)) — 



ksTix) 

1 
kBT{x) 



I dv 



Fv 



dt 



kBT{x) 

,2\ F 



kBT{x 



-dx 

(76) 

using V o dv ~ (l/2)(w' + v){v' — v) and vdt = dx, and 
noting that x is now deterministic, meaning the integra- 
tion rules are irrelevant. The second and third terms 
correctly reproduce the form of the change in medium 
entropy as heat transfer to the environment, equal to 
negative heat transfer to the particle (in agreement with 
the result found in stochastic energetics [32|), divided by 
the instantaneous temperature, and do so only by virtue 
of the consideration of odd and even variables. 

We can use this SDE to produce distributions of 
entropy production and verify relevant fluctuation 
theorems. To do so, however, requires knowledge of 
the solution to the Fokker-Planck equation, for which 
there is no simple analytical form. To proceed we 
restrict ourselves to the stationary state and utilise 
the expansion found in [3l[ and [Sj] which expresses 
the stationary solution as a series expansion about the 
overdamped distribution: 



P 



'ix,v) 



Afr. 



kBT{x) 



exp 



dx' 



F{x') 



2T:kBT{x) 



exp 



kBT{x') 
mv^ 



2kBT{x) 



(77) 



where J\f is determined by normalisation, such that 



p^\x, v) ^ p^*---(x, v) + ^(l/7)>.(x, v) 

i=i 

Pi{x, v) has a general form 

Pi{x,v) = 

ECi,k{x)Hk{v^m/kBT{x)) 
— /„ , ^, , = exp 



(78) 



^j2T:kBT(x) jni 



2kBT{x) 



(79) 



where constants a^, bi and functions Ci^^ix) are found by 
an iterative procedure, and Hk{y) are Hermite polyno- 
mials defined as 

-2 d'' ^y^ 



H,{y) = {-Ife- 



dy^ 



(80) 
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FIG. 1. Mean dimensionless entropy production, for example 
I, for a range of damping coefficients as predicted by a first 
order approximation in Eq. 1)84^ (solid line), an integral over 
the heat current, Eq. (|82p (dashed line) and a Monte Carlo 
average based on the SDE in Eq. (ff6)l (crosses) . 



Whilst the expansion has the formal deficiency that 
the expansion parameter is not unitless it suffices for 
a theoretical illustration where we can consider it in a 
limit where it is appropriate. We consider units fcs = 
T = m = 1, a harmonic confining potential such that 
F{x) = —X, a temperature profile 



1 



T{x) = 1 -f- - tanh(a;) 



(81) 



and approximate the stationary distribution by consid- 
ering the expansion in Eq. (|78p to fourth order in 7^^, 
applying the formalism numerically. 

We first demonstrate that this approach yields a 
result which maps onto the expected phenomenological 
expression for dimensionless internal entropy generation 



rf(A^tot) 
dt 



+00 



dx Jq{3l 



1 



d 



dx \kBT{x) 



(82) 



where Jq{x) is the stationary heat current defined as 



r+00 2^ 
Jq{x) = I dv -mv^p''\x,v). 



(83) 



Figure [T] shows the dimensionless entropy production 
obtained by performing the integral in Eq. (|82p using 
a numerically calculated p^*{x,v), compared with that 
obtained by averaging the SDE in Eq. ((76)) by Monte 
Carlo simulation of the underlying particle dynamics, for 
a range of damping coefficients, alongside a demonstra- 
bly positive first order approximation based on the first 
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correction term in Eq. (j78p given by 



d{ASu 



dt 



dx 



kBP^'^x) (dT{x) 



2m-fT{x) V dx 



where 



f\x) 



+00 



dv p''^{x,v). 



(84) 



(85) 



Our formalism for the rate of change of (AS'tot) agrees 
with Eq. ([5^ and both are consistent with Eq. ([M)) 
in the 7 — >■ 00 hmit. We point out that total entropy 
production decreases as coupling to the environment 
increases which may seem counter-intuitive, but we 
emphasise that with increased coupling, despite greater 
heat transfer to and from the environment, there is 
highly diminished spatial heat transport (the latter 
being the cause of entropy production) as the system is 
brought closer to a local equilibrium. 

We can use the SDE for entropy production (Eq. ((75)) ) 
to move beyond a classical description of mean entropy 
production to one described by Jarzynski, Scifcrt, 
Sekimoto and others [^, [2^, [33 where we can iden- 
tify entropy generating and destroying trajectories. 
We can explicitly calculate the distribution of total 
entropy production which is shown for 7 = 10 in 
Figure [2] for various process intervals, along with a 
demonstration that it adheres to an IFT throughout. 
Additionally, since we consider the stationary state we 
can demonstrate a detailed fluctuation theorem of the 
formp(AS'tot)/p(-A5tot) = exp(A5tot) [HI as shown in 
Figure H 

Finally we point out that, being in the stationary 
state, d{/S.S^)/dt = 0, but since it is a non-equilibrium 
stationary state that is asymmetric in the odd velocity 
variable we have A^a 7^ in detail, as is clear in 
Eq. ([M]) . We can demonstrate the increasing range of 
values of A^a as 7 is reduced and the system is taken 
further away from local equilibrium, with its symmetric 
velocity distribution, by generating the distribution of 
AS'3 using Eq. (|64p for a given time interval, as shown in 
Figure m Such a result highlights the fact that although 
a non-zero d{AS3)/dt is only possible during relaxation 
as shown by Eq. (p7|) . the specific evolution of AS3 for 
each trajectory is brought about by non-equilibrium 
constraints that cause the stationary solution to depart 
from equilibrium. 



VI. EXAMPLE II: PARTICLE DRIVEN BY A 
NON-CONSERVATIVE FORCE 

Once again utilising the full phase space Langevin de- 
scription of the dynamics, we consider diffusion of a par- 
ticle on a ring driven by a spatially independent non- 
conservative force and spatially independent (additive) 
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FIG. 2. Distributions of dimensionless total entropy produc- 
tion AS'tot, for example I, for 7 = 10 together with a demon- 
stration of adherence to an IFT. Distributions shown are for 
process intervals from t — 2 (narrowest) to i = 44 (widest) in 
steps of 6 units. 
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FIG. 3. Verification of a detailed fluctuation theorem for ex- 
ample I using data from simulation for 7 = 10 at time i = 8. 



noise such that 

dx — vdt 



F(t) 
dv = -jvdt H ^—dt 



2kBTj 



dW 



(86) 



thus giving A^^ = 0, Al'"' = v, A^ ^ ~jv, A^"" = 
F{t)/m, Dx = and Dy = ksTj/m. For any non-zero 
value of F{t) there will exist a stationary solution with 
an asymmetric Gaussian distribution in v and a uniform 
distribution in x due to the symmetry of the problem. 
Any relaxation from a given stationary state caused by 
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FIG. 4. Distributions of AS'a for example I evaluated at i = 8 
for a range of 7 from 7"^ = 0.1 (narrowest) to 7"^ = 0.4 
(widest). 



FIG. 5. Positive mean rates of dimensionless entropy change 
against time for example II, where we consider the transition 
between stationary states of a driven particle on a ring with 
Fo = 1, Fi = -1, io = 1 and fcs = m = 7 = T = 1. 



changes to the non-conservative force will then also result 
in a uniform distribution in x for all time by the transla- 
tional symmetry. As such we may proceed by consider- 
ing the marginalised velocity distribution when starting 
from a stationary state. Exploiting the fact that the ini- 
tial Gaussian solution will remain Gaussian for any F(t\ 
we can parameterise a transient solution to the Fokker- 
Planck equation 



p(a;, w,i) ex 



27rfcBT 



exp 



m(v — {v)y 



2kBT 



with 



dt 



7W 

m 



such that 



{vr = — 

7717 



(87) 



(88) 



(89) 



A scenario where closed form solutions exist for all contri- 
butions to entropy production is that of an instantaneous 
step change in the driving force F{t) so that we have 



F{t) = 



Fo 
Fi 



t< to, 
t>to, 



(90) 



and 



(«>W 



{Fi + e 



-fit-to) 



(Fo 



Fa/mj 
Fi))/to7 



t < to, 
t>to. 



(91) 



Performing the relevant integrals in Eqs. ([57)1 . ([55]) . ([CTjl 
and (|67p we then obtain 



dt 



d{ASi) 
di 



Fg/m-fkBT 
(i^o + Fi(eT(*-*")-l))' 



-27(t-to) 





{Fo^Fif/m-fkBT 



d{AS2) 
dt 



F^/m-fkBT 
F^/TwykBT 



t< to, 
t>to, 



t < to, 

t>to, 

(92) 
t<to, 

t>to, 

(93) 
(94) 



and 

djASs) 
dt 



-2e-'^(*-*")Fi(i^i - Fo)/mjkBT 



t < to, 

t>to. 

(95) 
Choosing the specific case of a reversal of the driving 
force such that it changes from i^o = 1 to Fi = — 1 at 
time to = 1 and employing units fc^ =m = 7 = T = l, 
we can generate the results shown in Figures [5] and [51 

We note first that the mean rates of change of 
all three contributions AS'tot, AS*! and AS2 are positive, 
refiecting their adherence to an IFT. All three mean 
rates of change are constant for t < to = 1, are per- 
turbed by the change in direction of the force, and relax 
back to constant values consistent with the transition 
between the stationary states. A key feature of this 
behaviour is that upon perturbation, the total entropy 
production rate decreases which would not emerge 
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FIG. 6. Unbounded mean rates of dimensionless entropy 
change for example II, the driven system on a ring with 
Fo = 1, Fi = -1, to = 1 and kB=m^'y = T=l. 



using an overdamped description of the dynamics. 
This feature can be explained by the existence of the 
d{AS3)/dt contribution to the mean entropy production 
rate, which may take negative values depending on 
the relationship between the instantaneous distribution 
and the stationary distribution. In this specific case, 
the large negative value for d{AS3)/dt indicates that 
upon reversal of the force the instantaneous distribution 
corresponds to particle motion, on average, in a direction 
counter to that expected to result from the new value 
of the force. The velocity distribution does relax, of 
course, to the distribution that corresponds to the new 
value of the force and so the mean rate of change of AS3 
decays away. An important point to draw from Figure [S] 
is that AS3, ASi + AS3 and AS'2 + AS3 cannot be 
expected, in general, to be positive, refiecting that they 
cannot be expressed in the form of Eq. ^ and thus do 
not obey IFTs. This means previous approaches where 
the entropy production can always be divided into two 
positive quantities |19l - [2l| and the house-keeping heat 
can be expected to obey an IFT [ij], do not extend to 
the systems considered here. 

We consider this example to be a helpful illustration of 
how entropy production cannot always be divided into 
two contributions which derive from relaxation, and an 
absence of detailed balance owing to a non-equilibrium 
constraint, respectively. Explicitly, the non-equilibrium 
constraint here is the constant force which produces en- 
tropy in the stationary state by inducing a constant fiux 
around the ring. The mean rate of entropy production 
in that stationary state is characterised by d{AS2)/dt 
which remains constant throughout the process owing 
to the constant magnitude of the force which is applied. 
However, both AS'2 and AS3 are non-zero only in the 



presence of a non-equilibrium constraint which breaks 
detailed balance. At the same time the mean rate of 
change of AS3 is non-zero only when the distribution 
is relaxing to a new stationary solution in the same 
manner as A5i. Whilst ASi describes the entropy 
production that arises from an evolution of the prob- 
ability distribution of a general set of variables, AS3 
expresses what ASi explicitly leaves out: the additional 
impact of relaxation on entropy production that relates 
to the a priori physical specification of the variables as 
odd or even. Clearly, given that the non-equilibrium 
constraint is a force of constant magnitude, reflected 
by the constant d{AS2)/dt, it is reasonable to consider 
the sum of ASi and AS3 as the contribution that arises 
due to relaxation to a new stationary state, particularly 
when the form of its mean rate of change in Figure [B] is 
contrasted with that of AStot, ASi and A52 in Figure 
[5] We may make the analysis complete by considering 
the SDEs for all contributions. The explicit Ito forms of 
Eqs. dni, (03), & and §^ are given as 

■ A^ ITT- , s , 771, ,,, d(v) , F(t) , 

(96) 

,A ^ 1 fFH) , A , 777 , , ,, d(v) , 

dAS, = j-^ ^-^ - Mv)j dv-j^iv- («)) -^dt 



dAS2 = -^^dv + P^dx 
-yksT ksT 



dAS3 



jksT 



dv 



(97) 
(98) 

(99) 



and illustrate the behaviour of all the contributions. 
dAStot is only zero when (tj) = 0, F ~ and d{v)/dt ~ 
meaning the system is in the equilibrium state. dASi is 
zero whenever (v) = F/rwy and d{v)/dt = correspond- 
ing to any stationary state, equilibrium or otherwise, 
whilst dAS2 and dAS3 contribute independently of 
properties of the distribution (namely (v)), but only 
when the non-equilibrium constraint is present such that 
F{t) 7^ 0. dASs however, has a mean contribution of 
zero at stationarity since (dv) = for any stationary 
state. We can calculate distributions of all the contribu- 
tions, as measured from the force reversal, numerically 
using the above SDEs and demonstrate the validity 
of IFTs, where appropriate, in Figures [7] and [H We 
observe that all distributions take Gaussian form, to be 
expected as the model is essentially a recasting of the 
overdamped dragged oscillator found in |36| where the 
further, but non-general, detailed fluctuation theorem 
symmetry p{AStot)/p{-AStot) = exp(AS'tot) has been 
noted to hold over finite times [Sa], but stressed else- 
where [331 to be coincidental. Further insight into this 
coincidence can be derived from the form of the SDEs 
which yield Gaussian distributions (for the given initial 
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FIG. 7. Distributions of entropy productions AStot (solid 
line), ASi (wide dashed line), AS2 (narrow dashed line) and 
ASs (dotted line) measured at times At = t — to = 1, At = 2, 
Ai = 3 and At = 4 after the reversal of the force for Fq = 1, 
Fi = —1, to = 1 and fc_B=jn = 7 = T=l. 



conditions) since they comprise only drift and additive 
noise terms (that is, no terms of the form f{v)dv). Such 
properties however, do not distract from the nature of 
the contributions which can be readily observed: the 
distributions in A^i and AS'3 develop fastest at first 
reflecting the initially fast response of the distribution to 
the change in force. However, distributions for both A52 
and A5'tot develop steadily, owing to their contributions 
being characterised by steady heat dissipation. As such, 
as time progresses, the distribution of AS*! ceases to 
develop as the system reaches the new stationary state 
and the distributions of A5'2 and A^tot continue to shift 
to the right until they eventually dominate. Similarly 
for AS'3, we observe here that the distribution stops 
evolving despite receiving non-zero contributions. 

For completeness we investigate the same model with 
a less trivial time dependence in the non-conservative 
force, along with its approach to the overdamped limit 
where such systems have been considered previously 
|2ll [38| . We employ the force protocol 



F{t) = 1.5 - 0.5 tanh(-5(t - 1)) 



(100) 



and perform the calculations numerically for two values 
of damping coefficient, 7 = 1 and 7 = 5. We point 
out again that the meaning of d{AS2)/dt for this system 
is easily elucidated since the non-equilibrium constraint, 
F(i), being phase space independent, leads to J"'^*- oc p^*" 
so that 



d{AS2) d{ASu 



dt 



dt 



mjkBT 
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FIG. 8. Illustration of adherence to IFTs by consideration 
of the average (exp[— AS]) against time, At = t — to after 
the force reversal, for A5tot, ASi and AS2 (indicated) and 
the failure to adhere to an IFT of AS3 (solid line, fourth 
subplot), ASi + AS3 (wide dashed line, fourth subplot) and 
A52 + AS3 (narrow dashed line, fourth subplot) for Fo = 1, 
Fi = —1, to = 1 and fcs=m = 7 = r=l. 
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FIG. 9. Mean rates of change of ASi (wide dashed line), AS2 
(narrow dashed line), AS3 (dotted line), and their sum A^tot 
(solid line) for example II with a time dependent force given 
by Eq. (|100|l with units ks — m — T — 1, and for 7 = 1 (top) 
and 7 = 5 (bottom). 



The mean contributions for such a protocol for two val- 
ues of the damping coefficient, again starting from the 
stationary state, are shown in Figure [9] Note that in 
this case the contribution d{AS3)/dt is positive, reflect- 
ing that as the non-conservative force decreases, the in- 
stantaneous distribution corresponds to a greater average 
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particle flux than would be expected from the instanta- 
neous value of the force, thus producing more entropy 
in the process of relaxation from one stationary state to 
the other than would be expected if the relaxation were 
instantaneous. As 7 increases, the asymmetry of the sta- 
tionary state (in velocity) decreases and the contribution 
from AS3 diminishes. Consequently, the two stationary 
distributions become increasingly similar, meaning the 
contribution AS*! also diminishes rendering the total en- 
tropy production almost entirely comprised of the con- 
tribution from AS'2. When the full overdamped limit is 
taken AS'2 is the only contribution and the results map 
onto those found in l2li . 



VII. DISCUSSION AND CONCLUSIONS 

We have derived SDEs describing the fluctuating 
evolution of three contributions to entropy production, 
along with expressions for their mean behaviour, and 
demonstrated that two of these contributions obey 
IFTs and thus are rigorously positive in the mean. 
Furthermore we have demonstrated that whilst these 
two naturally align themselves with the irreversibility 
associated with relaxation, and non-equilibrium con- 
straints, respectively, the inclusion of odd dynamical 
variables can give rise to a third term, which has no 
bounds on its sign and which cannot be so readily 
associated with one origin of entropy production or the 
other. We have sought to make these expressions as 
general as possible, within reason, with the intention 
that they may be applied to any system (physical or 
otherwise) described by stochastic differential equations, 
providing a framework for the discussion of entropy 
production, as defined here, within as wide a range of 
applications as may be relevant. To this end, we have 
considered a simple heat conduction problem, and after 
demonstrating that a full phase space representation of 
the dynamics is crucial to the treatment of its entropy 
production, we have examined specifically how it may 
be evaluated. The second example, that of a transition 
between stationary states of drift-diffusion on a ring, 
demonstrates the need for a third contribution to 
entropy production in the analysis, and provides some 
intuitive understanding of its nature. 

We suggest that the division of the total entropy 
production into ASi, AS2 and AS3, as we propose, is 
always helpful for three main reasons. The first is the 
ability to identify the physical origins of irreversibility in 
a process (relaxation and non-equilibrium constraints) 
and the interplay between them. The second is related to 
the identification of IFTs with the subsequent positivity 
requirements and restrictions on the statistics for the 
two contributions, ASi and AS'2, which unambiguously 
align themselves with the two causes of irreversibility. 
And thirdly, the more delicate reason that all three 
contributions are constructed from total path probability 



densities with equivalent measures such that they align 
themselves unambiguously with the same contributions 
found in master equation approaches which are neces- 
sarily formed from path probabilities |23j . This third 
point may be contrasted with an alternative division of 
the total entropy production into a system and medium 
contribution, neither of which can be expressed as ratios 
of total path probabilities. Being formed from a proba- 
bility density, the system entropy, as defined in [ll| and 
employed here (Eq. ([T5|) implies Sgys = —\np{x{t),t)), 
is strictly not dimensionally correct (even if one argues 
that the relative entropy change is well defined [23 ) , but 
moreover does not share the same form as the system 
entropy that appears in master equation approaches, 
which if followed would imply Sgys ~ ~ liip(a;(t), t)dx{i). 
Such ambiguity then also enters into the definition of 
the medium entropy, but can be avoided altogether by 
considering the total entropy production in terms of 
explicit measures of irreversibility such that AStot is 
comprised of ASi, AS2 and AS3, all four of which do 
not suffer from such issues. 

Generalisations that are immediately obvious be- 
yond the description given here may include correlated 
stochastic processes for which the method described 
should prove suitable, and processes where we no longer 
assume D{ex) = D{x) for which the generalisation may 
be more challenging. One might note, however that 
processes which do not possess such a symmetry are 
unlikely to be physically meaningful. Furthermore, it 
would be natural to explore examples involving odd 
dynamical variables such as angular momentum and 
magnetic dipole moments, and to include driving by 
external forces that are themselves odd under time 
reversal, such as torques and magnetic fields. We 
expect to find further richness in the phenomenology of 
entropy production associated with stochastic dynamical 
behaviour. 
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Appendix A: The use of short time propagators Avith 
multiplicative noise 



Here we consider one of the terms in Eq. ([M)) and de- 
rive Eq. (gS]). By utihsing Eqs. ^^ and dl?]) we can de- 
scribe an increment in the medium entropy production 
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according to the formalism of Scifert [ll[ as 



dAS'nicd — 

^ilnA(r')-ilnA(r) 



dxf 



dxf 



2'" ''■ ' 2'" '^' ' ' 4:D,{r')dt 4:D,{r)dt 
dx, ( Af''{r) A'^{r) Af(r') Af^{r') 



2 V D,{r) D,{r) D,{r') D,{r') 
1 dD^ir) ^^ 1 c)A(r') 



-2a- 



-2b- 



D,{r) dn D,{r') dr[ 

dt ^(Ar(r) + Ar(r))2 (Ar(r')~AHr'))2 
4 



A(r) 
-adtlD.ir)^^'^^'^ 



Diir') 

d fAf^ir) 



D^{r'} 



Mr-<xmy-<,i: 



2 . / 5'A(r) 
-a at ' 



9rf 




Ddr) V 5^' 



^,^Jd'D,ir') 1 ^aA(r^)^ 



ar',2 A(r^) V ar',^ 



(Al) 



matrix, wc may write 

fir) = fiil-a)x+ax') 
^(l-a)/(a;) + a/(a/) 
= f{x) + a{f{x')-f{x)) 
= fix) + adfix) 

, (df{x) 



dt+Vf{x) -dx+Y^ ^^^V^f{x)dt 



f{x)+a 




dt + Y,]-B,{xfVf{x)dt 



■J2^^ iMx)dt + B,{x)dwA . 



(A2) 

Considering all instances of multiplication along with the 
definitions of r, r', a and b, we find the following heuristic 
rules 



f{r)dt = f{x)dt + 0{dt^^^) 



f{r')dt = f{x)dt + 0{dt^^^) 



(A3) 



(A4) 



fir)dx, = fix)dx, + 2aD,{x)^^dt + 0{dt^/^) (A5) 

OXi 



where time dependence in variables A"', A''°^ and Di 
is assumed, but not explicitly written for brevity. We 
may proceed by understanding that the quantity dxi 
is an increment in an underlying SDE, meaning that 
we must consider all multiplications of the form f{r)dx 
as infinitesimal stochastic integrals with a summation 
rule defined by the evaluation point r. For example, 
r = X would imply an Ito integration, r = (1/2) (x + x') 
would imply Stratonovich and so on. To consolidate the 
above it is sensible to convert all multiplications into one 
type, for which we choose Ito in order to apply the Ito 
stochastic calculus transparently using the heuristic rules 
(dWi)"^ = dt and dWidWj = 0, and to drop all terms of 
order dt^'^ and higher. To do so we apply the following 
reasoning. For a suitably smooth function, /(r), and for 
infinitesimal dt, with r constructed from x and a/ using 
a parameter a, and for the case of a diagonal diffusion 



f{r')dx, = fix)dx, + 2(1 - b)D,{x)^^dt + 0(dt3/2) 

OXi 

(A6) 
giving us a method for converting all multiplications into 
Ito form. We use a similar reasoning to approximate 

D,{r) ^ D,{x) + adiD.ix)) (A7) 

D,{r') ~ D,{x) + (1 - b)d{D,{x)) (A8) 

which along with the approximations to second order in 
dxi, and therefore d{Di{x)), of the following form: 

(1 + d{D,{x)))-^ ~ 1 - d{D,{x)) + d{D,{x)f (A9) 

d{D,{x)f 



Ml + d{D,{x))) ~ d{D,{x)) 



(AlO) 



and an Ito definition of d{D{x)), allow us to write the 
first four terms in Eq. (IA1[) to first order in dt as 



dx^ 



dxf 



ilnA(r^)-ilnA(r) + ^^^^^-^^^^ 



iil-b)^-a^) f dD,{x) 
2D,{x) \ dx^ 



dt. 



(All) 
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Using the above, and the heuristic rules in Eqs. (|A3IIA6p . 
we obtain 






dt 



dx, 



dxi 



Di{x) dxi Di{x) dxi 

1 diMx), 

[a+b) „ , , — dx,. 



D,{x) dxi 



{b^-2h-a^ 



d^D.,{x) 1 r dP^jxY 

dxj Di{x) \ dx, , 



2D,{x) V dxi 



dt 



dt 



(A12) 



which depends on the choice of a and 6. We note, 
however that without niultiphcative noise (ie. for 
{d/dxi)Di{x) = 0), where the inherent mathematical 
ambiguity in stochastic integrals is absent, the depen- 
dence on evaluation points (a and 6) disappears. So this 
dependence is evidently related to the ambiguity of eval- 
uation point in a stochastic integral. However, the un- 
derlying SDEs and entropy arc not, and should not be, 
ambiguous since we have specified Ito SDEs and have 
used the short time propagator appropriate for their cor- 
responding Fokker-Planck equation. Since all evaluation 
points lead to the correct path probability density we are 
not obliged to consider, for example, only Ito-type mul- 
tiplication (a = & = 0) simply because the underlying 
SDEs arc of Ito form. Rather, to proceed we recognise 
that with multiplicative noise we must ensure that we 
evaluate the two transition probability densities at pre- 
cisely the same coordinates and not just at same time 
(which would suffice for additive noise), for the same 
reasons that make stochastic integration sensitive to the 
specific integration scheme when the integrand has de- 
pendence on the integrating variable. Alternatively it 
may be reasoned that as dt — >■ we require the short 
time propagators to approach jump transition probabili- 
ties of a master equation. Under such a description the 
entropy production can be unambiguously described by 
the ratios of probabilitities appearing in a master equa- 
tion approach [13, [23. Therefore, if we represent such a 
quantity using the short time propagators we require the 
transition rates in both numerator and denominator to 
be evaluated equivalently. Since these are characterised 
by our system variables Ai, Bi etc, to effect such a condi- 
tion we require r' = r which is equivalent to making the 
choice 6 = 1 — a, noting that a is still a free parameter. 
One may think of this as insisting that the path transfor- 
mation ^ (t) = e£(r — t) persists on a (sub) infinitesimal 
scale, which would not matter in normal calculus, so that 
the noise is experienced in precisely the right way. Insert- 
ing 6 = 1 — a into the above yields Eq. (pS)) which has 
no dependence on the choice a, an indication that it is 



defined in a sound fashion. 



Appendix B: Consistency of entropy contributions 

By construction, we have AS'tot = AS*! + AS'2 + A^a 
which according to the expressions in Eqs. ((571) . f55)) . (pT|) 
and (j67p means we require 
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Di{x) \ p^^{ex) I 
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Thus in order that everything is consistent we require 
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(B2) 

By substitution of the definitions of the fluxes this re- 
duces to 



= ^ I dx 



2A'r(a;)M^ _ ^^Diix) dp{x) 



dxi 



dxi dxi 



d{D,{x)p{x)) j^n^^w 

+ p{x)D,{x){(<j,[{x)f + {<j,[{ex)f) 



2p{x)- 



dxt 



[(x) + £,cl>'^{ex)) 



+ p{x){Af{x)+A^''^xM{x) 
+ e,p{x){Anx)-Ar (xMiex) 



(B3) 

Integrating by parts, dropping or cancelling boundary 
terms and using the definition of the irreversible and re- 
versible drift terms yields the condition 



= E/^^p(^) 



^ dA^jx) ^ ,d-'Ddx) 



dxi ' dxj 
D,{x) {c^Ux) + <^'l{ex)) + D,{x) ((0^^))' + mex}f) 
2^^ mx) + e,c^',{ex)) + {j^ix) + A^ {x))c^',{x) 



+ {^aex) + A'r{ex))cp[{ex) 



(B4) 
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The divergencelcss stationary distribution condition how- 
ever yields 

0=(- {A^{x) + Al'-ixMix) - D,{x){ct>[{x)f 
dA'^ix) dAf^{x) d^D,{x) 



dxi 



dxi 



dxj 
,dD,{x)^,, 



Combining the two conditions in Eqs. (|B5p and 
yields the contents of the brackets in equation (|B4 
so the result is proved. 



IB6D 

and 



+ D,{x)4>':ix)+2':^^cl^[{x)\e-^('^\ (B5) 



but also 



= 



- i^liex) + Ar{exM,{^ex) - D,iexM{ex)f 
dA^'iex) 9Af^(£a;) d^Di{ex) 



d{£i 



d{£iXi 



d{£iXiY 



.D,{exWl{ex)+2^-^^cp[{ex)\e- 



(B6) 
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